AFIT/GSO  /  ENP /93D-03 


AD-A273  837 

inniniii 


DTIC 

SELECTE 
DEC  16 1393 

A 


OBJECTIVE  IMAGE  QUALITY  METRICS: 
APPLICATIONS  FOR  PARTIALLY  COMPENSATED 
IMAGES  OF  SPACE  OBJECTS 

THESIS 
David  J.  Lee 
Captain,  USAF 

AFIT/GSO/ENP/93D-03 


93-30515 

IMlf«  ■■•jl  _ 


Approved  for  public  release;  distribution  unlimited 


98  12  16129 


AFIT/GSO/ENP/93D-03 


OBJECTIVE  IMAGE  QUALITY  METRICS; 
APPLICATIONS  FOR  PARTIALLY  COMPENSATED 
IMAGES  OF  SPACE  OBJECTS 


THESIS 


Presented  to  the  Faculty  of  the  Graduate  School  of  Engineering 
of  the  Air  Force  Institute  of  Technology 
Air  University 
In  Partial  Fulfillnaent  of  the 
Requirements  for  the  Degree  of 
Master  of  Science  in  Space  Operations 


David  J.  Lee,  B.Sci.,  Electrical  and  Electronics  Engineering 

Captain,  USAF 


December  15,  1993 


Accer.icr  For 

NTIS 

DTIC 

Ur.jnn 

CRA&I  ^ 

TAB  □ 

1  By  . _  _  .. 

Di  jtrib 

JliOf  / 

Avr.iiaoiluy  Codes 

Oist 

^^1 

Avail  r)':dlOr 
Special 

Approved  for  public  release;  distribution  unlimited 


Preface 


The  most  important  lesson  a  student  can  learn  during  a  thesis  effort  (besides 
time  management)  is  teamwork.  The  most  important  example  of  this  in  my  thesis 
research  is  the  teamwork  between  my  wife  and  myself.  My  contribution  to  the  team 
effort  consisted  of  schoolwork.  Denise’s  contribution  to  the  team  effort  consisted  of 
everything  else:  balancing  the  checkbook,  paying  bills,  taking  the  car  to  the  garage, 
mowing  the  lawn,  cleaning  the  house,  and  all  the  other  things  I  didn’t  have  time  to 
do.  She  already  knows  how  much  I  appreciate  her  sacrifices  over  the  last  months, 
and  now  you,  the  reader,  know  as  well. 

The  other  important  example  of  teamwork  in  this  effort  is  between  myself  and 
my  advisor.  Dr.  Mike  Roggemann.  I  felt  fortunate  that  I  was  independent  enough 
to  make  this  thesis  my  own,  yet  felt  I  had  ample  guidance  when  running  up  against 
the  inevitable  brick  wall.  Also,  hopefully  Dr.  Roggemann  knows  how  much  my 
classmates  and  I  appreciate  him  opening  “Father  Roggemann’s  Home  for  Wayward 
Sp<ice  Operations  Students”  by  euivising  so  many  out-of-department  theses. 

Thanks  aJso  to  Drs.  Welsh  and  Chan  for  their  help  in  my  research  effort. 

David  J.  Lee 
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Abstract 

Digital  image  reconstruction  tasks  currently  require  human  intervention  for  a 
subjective  evaluation  of  image  quality.  A  method  for  un-supervised  measurement 
of  digittil  image  quality  is  desired.  This  research  investigated  various  parameters 
(metrics)  that  can  be  automatically  extracted  from  a  digitcil  image  and  tested  how 
well  they  correlated  with  image  quality.  Specihcally,  images  of  orbiting  satellites 
captured  by  a  partially  compensated  adaptive  optics  telescope  were  dealt  with. 

Two  different  types  of  quantities  were  investigated:  1)  Fourier  spectral  pa¬ 
rameters,  based  on  the  spatial-frequency  sensitivities  of  the  HVS;  and  2)  Histogram 
shape  parameters  (i.e  image  statistical  moments)  giving  quantitative  insight  into  the 
structural  content,  information  content,  and  brightness  distribution  of  an  image. 

An  atmospheric  imaging  simulator  was  used  to  generate  a  test  database  of  im¬ 
ages.  The  use  of  .simulated  imagery  allowed  precise  control  of  the  imaging  parameters 
directly  relating  to  image  quality:  1)  Root  Mean  Square  Error;  2)  Seeing  conditions 
(Fried  Parameter,  r,,);  and  3)  Target  magnitude.  This  in  turn  allowed  quantitative 
testing  of  candidate  image  quality  metrics.  Metrics  could  also  be  tested  agmnst 
the  user  defined  parameters  of  the  reconstruction  process,  as  a  proof-of-concept  for 
totally  un-supervised  image  reconstruction.  Finally,  based  on  this  testing,  two  suc¬ 
cessful  image  quadity  metrics  are  recommended. 

Applications  that  require  automatic  evaluation  of  relative  image  quality  should 
prove  feasible.  Applications  of  interest  to  the  thesis  sponsor  include:  1)  “sifting” 
large  numbers  of  image  frames  to  exclude  “bad”  images  from  further  processing; 
and  2)  selecting  the  best  reconstructed  image  to  forward  to  customers. 
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OBJECTIVE  IMAGE  QUALITY  METRICS: 
APPLICATIONS  FOR  PARTIALLY  COMPENSATED 
IMAGES  OF  SPACE  OBJECTS 


I.  Introduction 

1.1  Background 

This  research  is  concerned  with  the  measurement  of  the  '“qucility”  of  a  digital 
image,  specifically  the  digital  images  captured  and  processed  at  the  Air  Force  Maui 
Optical  Station  (AMOS),  the  sponsor  of  this  thesis.  As  background  and  motivation, 
this  section  will  present  an  outline  of  the  theory  and  principles  of  AMOS  operations 
which  are  relevant  to  the  problem  of  image  quality  measurement. 

1.1.1  Imaging  of  Orbiting  Sateilites.  AMOS  is  located  atop  Mt  HaJeakala 
on  the  island  of  Maui  in  the  state  of  Hawaii.  The  station  operates  a  1.6  meter 
telescope  used  to  obtain  high  resolution  imagery  of  space  objects,  especially  low 
earth  orbit  (LEO)  satellites.  [13]. 

Since  AMOS  is  interested  in  imaging  versus  merely  tracking  a  satellite,  the 
resolving  power  of  the  telescope  is  aJl-important.  A  1.6  meter  telescope,  in  a  vacuum, 
has  a  diffraction  limited  (or  theoretical)  resolution,  0,  of  about  0.08  arc  seconds  or 
about  .39  microradians  (if  we  use  Rayleigh’s  criterion,  0  -  1.22A/£>  where  A  is  the 
wavelength  of  light  being  detected,  say  500  nanometers,  and  D  is  the  1.6  meter 
diameter  of  the  telescope).  For  a  satellite  500  kilometers  away  from  the  telescope, 
this  means  that  one  could  not  expect  to  discriminate  any  feature  smaller  than  19 
centimeters  from  any  adjacent  feature  on  the  satellite. 
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The  resolving  power  of  the  telescope  is  further  limited  by  the  intervening  at¬ 
mosphere  between  target  and  telescope.  Due  to  the  thermodynamics  of  the  Earth, 
the  atmosphere,  and  their  interaction,  the  atmosphere  is  not  thermcdly  uniform  and 
isotropic.  Atmospheric  turbulence  in  the  line  of  sight  of  the  telescope  causes  the 
index  of  refraction  of  the  atmosphere  to  vary  randomly  at  different  points  across  the 
field  of  view  of  the  telescope.  This  in  turn  leads  to  random  phase  and  amplitude 
aberrations  in  the  pupil  of  the  telescope  [5]. 

There  are  regions  of  relatively  constant  index  of  refraction  at  random  positions 
in  the  field  of  view  of  the  telescope,  lasting  for  a  random  length  of  time  (on  the  order 
of  microseconds).  The  view  is  filled  with  these  regions,  known  as  turbulent  eddies,  and 
the  statistics  of  the  sizes  and  durations  of  these  eddies  have  the  effect  of  limiting  the 
effective  diameter  (for  resolving  purposes)  of  the  telescope.  This  effective  diameter, 
known  as  the  Pned  Parameter  or  atmospheric  coherence  diameter,  r^,  depends  only 
upon  atmospheric  conditions,  regardless  of  the  physical  diameter  of  the  telescope  [5]. 
At  the  AMOS  location,  To  can  typically  vary  from  5  to  12  centimeters. 

Even  though  the  telescope  has  an  actual  diameter  of  1.6  meters,  and  can  gather 
photons  more  effectively  than  a  smaller  telescope,  with  conventional  optics  it  can  only 
resolve  space  objects  as  well  as  an  rg-diameter  telescope.  Since  To  is  an  order  of  mag¬ 
nitude  smaller  than  1.6  meters  (say  10  centimeters),  the  denominator  of  Rayleigh’s 
resolution  formula  becomes  smcdler  and  6  becomes  larger  by  the  same  factor.  For 
our  example  satellite  at  500  kilometers  slant  range,  we  can  now  only  resolve  objects 
on  the  order  of  a  meter  or  more  in  size. 

Another  complicating  factor  is  that  LEO  satellites  move  rapidly  across  the  sky, 
so  the  target  is  constantly  being  viewed  through  different  parts  of  the  atmosphere 
and  different  sets  of  turbulent  eddies,  each  with  different  statistical  properties.  Also, 
the  aspect  angle  between  the  sun,  satellite,  and  telescope  is  constantly  changing,  and, 
if  the  satellite  is  tumbling,  spinning,  or  tracking  the  earth,  it  presents  a  constantly 
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changing  aspect  to  the  telescope.  This  limits  the  length  of  exposures,  and  limits  the 
number  of  exposures  which  can  be  averaged  together  for  post-processing  [16]  [18]. 

1.1.2  Hybrid  Imaging.  There  is  nothing  that  can  be  done  about  the  trans¬ 
lation  and  rotation  of  a  target  satellite.  But  there  are  two  possible  approaches  to 
tackling  the  problem  of  atmosphere-induced  aberrations  when  imaging  space  objects: 

•  the  incoming  distorted  wavefront  can  be  physically  corrected  by  an  adaptive 
optics  system 

•  the  raw  telescope  images  can  be  digitally  reconstructed 

1.1. 2.1  Adaptive  Optics.  An  adaptive  optics  system  (Figure  1.1)  at¬ 
tempts  to  physically  correct  the  phase  aberrations  induced  by  the  atmosphere.  It 
uses  a  wavefront  sensor  which  measures  the  deformations  that  the  wavefront  has 
experienced  while  traveling  through  the  atmosphere.  The  controlling  computer  uses 
this  data  to  generate  commands  to  the  deformable  mirror,  which  ideally  should  in¬ 
duce  the  exact  conjugate  of  the  phase  aberrations  in  the  wavefront,  thereby  removing 
the  deformations,  in  real  time. 


Figure  1.1  Adaptive  Optics  Concept 

The  AMOS  Compensated  Imaging  System  (CIS)  (Figure  1.2)  is  a  typical  ex¬ 
ample  of  such  a  system  [13].  The  CIS  consists  of  a  closed-loop  system  with  wavefront 
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sensor,  wavefront  computer,  wavefront  corrector,  and  image  sensor.  The  wavefront 
sensor  uses  two  shearing  interferometers  to  measure  the  slope  of  the  wavefront  in  two 
orthogonal  directions  at  152  discrete  subaperture  positions.  The  wavefront  computer 
then  uses  these  slope  mecisurements  to  calculate  the  corrections  necessary  to  remove 
the  phase  distortions. 

The  wavefront  corrector  consists  of  a  tilt  corrector  mirror  and  a  deformable, 
monolithic,  piezoelectric  mirror  (DM)  with  168  actuators  [13].  The  tilt  corrector 
mirror  uses  commands  from  the  wavefront  computer  to  remove  gross  wavefront  tilt 
error,  “ccntroiding”  the  image  (keeping  it  centered  on  the  image  plane).  The  DM 
actuators  receive  commands  from  the  wavefront  computer  and  deform  the  shape  of 
the  mirror  to  at  lecist  partially  cancel  out  the  estimated  higher  order  errors.  A  slow- 
scan  charge-coupled  device  (CCD)  camera  then  captures  the  compensated  image  and 
records  it  on  a  computer  disk  and  then  digital  tape  for  eventual  digital  processing 
and  shipment  to  customers. 


Figure  1.2  AMOS  Compensated  Imaging  System 
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The  adaptive  optics  reconstruction  process  does  not  provide  perfect  correction 
of  an  image.  The  ^wlaptive  optics  process  is  limited  in  its  effectiveness  by: 

•  Spatial  density  of  wavefront  sensor  subapertures 

•  Spatial  density  of  deformable  mirror  actuators 

•  Errors  and  noise  in  wavefront  sensor  measurements  (sensitive  to  low  light  levels) 

•  Time  lag  between  wavefront  sensor  measurements  and  deformable  mirror  ac¬ 
tuation 

•  Amplitude  perturbations,  which  cannot  be  corrected  [1] 

•  Errors  in  telescope  construction  such  as  fixed  aberrations,  optical  misalign¬ 
ment,  and  actuator  misplticement. 

These  facts  leads  one  to  consider  supplementing  the  adaptive  optics  process  with 
image  reconstruction  via  digital  post-processing.  ^ 


1.1. 2.2  Digital  Image  Reconstruction.  Optical  imaging  with  discrete 
detectors  can  readily  be  treated  as  a  two-dimensional  linear  systems/discrete  signal 
processing  problem.  Linear,  shift-invariant  systems  are  completely  characterized  by 


f(x.y) 

Input 

"Actual”  Intensity 
Function  of  Target 


h(x,y) 


Impulse  Response  of 
Atmosphere/Telescope  System 


Output 

Intensity  Function 
at  Telescope  Detector 


Figure  1.3  Linear  System  Model 


their  impulse  response  or  point  spread  function  (PSF)  h{x,y).  The  output  of  such 


^Some  terminology:  an  adaptive  optics  system  that  has  less  than  one  WFS  subaperture  and 
DM  actuator  pair  per  atmospheric  coherence  diameter  such  as  the  AMOS  CIS,  is  said  to  be 
partially  compensated. 
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systems,  g{x,y),  is  related  to  the  input,  f{x,y),  by  the  convolution  sum  (6): 


9{^,y)  =  -  i>v-3)  =  f{x,y)*Hx,y),  (i-i) 

‘  1 

and  also  by  the  Fourier  Transform  ^  frequency  domain  filtering  property  or  convolu¬ 
tion  theorem: 


{g{^,y)  =  fi^,y)  *  <=>  {G{u,v)  =  f{u,v)H{u,v)),  (1.4) 


where  the  asterisk  denotes  convolution  and  denotes  a  Fourier  Transform  pair. 

The  following  terminology  is  adopted. 


•  H{u,v)  =»  Optical  Transfer  Function  (OTF) 

•  |^f(u,u)|  Modulation  Transfer  Function  (MTF) 

•  G{u,v)  Image  Spectrum 

•  |G(u,v)j  =>  Magnitude  Spectrum 

•  |(j'(u,v)p  =>  Spectral  Density. 

Note  that  in  this  formulation  the  frequency  variables  refer  to  spatial  frequency,  and 
high  spatial  frequency  generadly  corresponds  rapid  changes  in  image  gray  levels  spa¬ 
tially  across  the  image.  This  in  turn  often  corresponds  to  the  highly  resolved  features 
of  the  image. 


^The  2- Dimensional  Discrete  Fourier  Transform  is: 
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(for  square  arrays),  and  its  inverse  becomes 
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(1.3) 


u  and  V  arc  spatial  frequency  variables. 


The  imaging  process  can  be  viewed  as  a  linear  system.  The  actual  intensity 
function  f(x,y)  of  the  target  satellite  can  be  viewed  as  the  input.  The  intensity 
function  at  the  image  plane  of  the  telescope  (as  sampled  by  the  detectors)  is  the 
output  g{x,y).  The  diffraction  effects  of  the  telescope  and  the  “degradation”  effects 
of  the  atmosphere  are  characterized  by  a  point-spreaxi  function,  h{x,y). 

If  we  knew  the  PSF  of  the  imaging  process,  h{x,y),  we  could  ideally  find  the 
original  target  f{x,  y)  by  F{u,  v)  =  This  concept  is  known  as  inverse  filtering. 

There  are  three  problems  with  this  simple  approach: 


1.  U  H{u,  v)  approaches  zero  at  any  point  the  calculated  F{u,  v)  would  be  theoret¬ 
ically  unbounded  and  numerically  wrong,  a  manifestation  of  the  fact  that  there 
is  no  information  available  from  the  spatial  frequencies  where  H{u,v)  =  0. 

2.  Random  noise  is  always  introduced  in  the  detection  process,  adding  uncertainty 
to  the  measurement  of  g{x,y)  (See  Figure  1.4). 

3.  The  OTF  is  actually  a  random  variable,  changing  with  time,  because  of  noise 
and  the  constantly  changing  atmospheric  turbulence.  The  best  that  can  hoped 
for  is  knowledge  of  the  statistics  of  the  OTF. 


f(x,y) 


Input 
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impulse  Response  of 
Atmo^heie/Telescope  System 


Intensity  1 
at  Telescope  Detector' 
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Figure  1.4  Linear  System  Model  With  Detection  Noise 


Problems  with  noise  and  amplitude  aberrations  can  be  particilly  overcome  by 
averaging  large  numbers  of  short  exposures  together.  Intuitively  this  makes  some 
sense  since  a  sudden  random  change  (i.e.  noise)  in  one  sample  of  a  random  variable 
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would  be  partially  averaged  out  by  other  samples  taken  at  different  times.  The  net 
effect  would  be  a  “smoothing”  of  noise  effects.  An  estimate  of  F{u,  v)  can  be  obtained 
by  F{u,v)  =  where  (•  •  •)  denotes  ensemble  averaging.  An  example  ensemble- 

averaged  satellite  iraage^,  or  {g{x,y)),  is  shown  in  Figure  1.6.  The  magnitude  of  the 


Figure  1.5  Ensemble- Averaged  Satellite  Image 
Fourier  Transform  of  this  image,  or  magnitude  spectrum,  is  shown  in  Figure  1.6. 


Figure  1.6  Modulus  of  FFT  of  Satellite  Image 

^Since  AMOS  images  are  captured  on  a  two-dimensional  CCD  array,  it  is  customary  to  view 
an  image  as  a  two-dimensional  array  of  gray  levels.  AMOS  images  are  generally  processed  as  128 
by  128  matrices  of  gray  levels  with  14  bit  quantization.  In  other  words,  the  minimum  element 
(0)  is  black  and  the  maximum  element  (1G383)  is  white,  with  integer  gray  levels  linearly  scaled 
in  between  these  two  extremes.  The  value  of  a  gray  level  corresponds  to  a  measured  integrated 
average  intensity  at  a  corresponding  area  in  the  image  plane  of  the  telescope. 
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An  estimate  of  the  impulse  response  (PSF)  of  the  imaging  process  is  eilso 
sought.  In  optical  signal  processing,  a  point  source  of  light  can  serve  as  an  impube 
function.  The  sky  is  full  of  stars  which  serve  as  excellent  point  sources.  An  ensemble 
average  of  images  of  a  star  taken  immediately  before  or  after  a  target  satellite  pass 
provides  {h{x,y)),  an  estimate  of  the  PSF.  See  Figure  1.7  for  an  example.Taking 


Figure  1.7  Ensemble- Averaged  Star  Image  (Point  Spread  Function  Estimate) 

the  Fourier  Transform  of  the  average  PSF  gives  us  the  average  OTF  of  the  imaging 
process  (estimated)  (5).  See  Figure  1.8.  The  origin  is  in  the  center  of  the  2-D  surface 
plot. 


Figure  1.8  Magnitude  of  the  Ensemble- Averaged  OTF 

The  satellite  FFT  is  divided  by  the  OTF  estimate  above.  This  has  the  effect  of 
amplifying  the  higher  spatial  frequencies  of  the  satellite  image.  Notice,  however,  that 
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the  OTF  gets  smaller  with  increasing  spaticil  frequencies.  While  it  is  true  that  the 
presence  of  Fourier  coefficients  at  higher  spatial  frequencies  generally  implies  higher 
resolution,  the  telescope  has  a  fundamental  limit  on  its  resolution.  Therefore,  it  is 
band-limited  in  the  Fourier  domain.  Any  Fourier  coefficients  beyond  this  frequency 
can  only  be  the  result  of  noise  since  this  information  could  not  have  possibly  been 
captured  by  the  band-limited  telescope  system. 

The  higher  spatial  frequencies  of  the  telescope  passband  contain  valid  image 
information,  but  also  contmn  noise  energy  as  well.  Consider,  for  example,  if  the 
OTF,  H{u,v),  is  known  exactly,  but  G{u,v)  is  detected  in  the  presence  of  additive 
noise.  Applying  the  inverse  filtering  relation,  F(u,  v)  =  this  equation  would 

reduce  to  F{u,  v)  =  FyU,  v)  -f  where  N{u,  v)  is  the  image  noise  spectrum.  If 

the  OTF  ever  approaches  zero  in  the  telescope  passband,  which  it  does  (Figure  1.8), 
the  term  would  take  on  large  values  and  amplify  the  noise  in  the  image. 

It  is  reasonable  to  suppose  that  if  a  second  filter  is  applied,  one  which  attenuates 
the  effects  of  these  noise-dominated  higher  frequencies,  acceptable  results  might  be 
obtained.  AMOS  personnel  prefer  a  triangular  cross-section  (conical)  lowpass  filter 


Figure  1.9  Inverse  Filtering  Noise  Filter 
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for  reconstruction  noise  attenuation  (18)  (Figure  1.9).  This  modified  approach  to 
inverse  filtering  is  commonly  used  for  reconstructing  AMOS  images.  The  spectrum  of 
the  inverse  filtered,  triangle  filtered  example  image  is  shown  in  Figure  1.10.  Compare 
this  spectrum  to  the  original  ensemble  image  spectrum  (Figure  1.6).  Note  that  the 
new  inverse  filtered  spectrum  has  energy  content  in  the  higher  spatial  frequencies. 
The  inverse  Fourier  TYansform  of  this  spectrum  is  shown  in  Figure  1.11.  Compare 
this  reconstructed  image  to  the  original  ensemble  in  Figure  1.5 


Figure  1.10  Reconstructed  Image  Spectrum 


Figure  1.11  Reconstructed  Satellite  Image 
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l.S  The  Image  Quality  Problem 

It  is  gener2dly  accepted  that  the  ultimate  arbiter  cf  image  quality  is  the  sub¬ 
jective  opinion  of  the  user  of  an  image.  This  fundamental  axiom  implies  several 
problems  for  the  automatic  processing  of  large  amounts  of  AMOS  digital  imagery: 

•  Ensemble  Sifting 

During  a  particular  AMOS  satellite  imaging  session,  hundreds  of  raw  image 
frames  are  ingested  into  the  postprocessing  system  for  reconstruction.  Experi¬ 
ence  has  shown  that  some  image  frames  are  of  noticeably  poorer  quality  than 
others  in  the  ensemble  [22].  It  may  be  desirable  to  exclude  these  lower  quality 
images  from  the  reconstruction  ensemble  (21). 

•  Reconstructed  Image  Selection 

Recall  the  triangular  cross-section  noise  filter  of  the  modified  inverse  filtering 
reconstruction  method.  The  optimum  width  that  this  filter  should  have  is  not 
readily  apparent.  The  actual  bandwidth  of  the  opticcd  system  is  changing  ran¬ 
domly  with  To  and  the  performance  of  the  adaptive  o)  .*ics  system.  If  the  filter 
is  too  wide,  noise  wU!  be  let  in  which  wiU  detract  from  the  image  restoration 
goal.  If  the  filter  is  too  narrow,  legitimate  resolution  detail  from  the  target 
satellite  will  be  excluded  from  the  final  product.  Which  filter  width  should  be 
used?  Also,  several  reconstructed  images  are  produced  during  one  observing 
session.  The  customer  usually  requires  only  a  subset  of  these  images. 

These  two  classes  of  problems  could  be  handled  in  an  un-supervised  fashion  if  some 
sort  of  “quality  number”  could  automaliccdly  be  assigned  to  the  images.  This  number 
could  be  thresholded  for  rejection  or  <icceptance  of  images,  based  on  qudity. 

1.3  Research  Objective 

The  gocil  of  this  research  is  to  find  a  way  to  objectively  and  automatically 
measure  the  quality  of  an  AMOS  digitcil  satelUte  image.  This  research  will  inves- 
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tigate  if  there  exist  certain  quantities  which  can  be  extracted  from  digital  images 
of  space  objects  that  correlate  with  perceived  image  quality.  These  parameters  will 
be  obtained  using  the  Interactive  Data  Language  (IDL),  a  commercicdly  available 
data  processing  software  package  which  is  used  to  process  imagery  operationally  by 
the  thesis  sponsors  at  AMOS  [14]  [15].  Any  of  these  parameters  which  prove  to  be 
successful  may  eventually  be  integrated  into  AMOS  operaiional  image  processing 
software  wherever  some  indication  of  relative  image  quedity  is  required. 

1.4  Scope 

This  thesis  deals  with  quantitative  post-processing  algorithms  for  monochrome 
digital  images  of  extended  objects  against  “dark”  backgrounds,  as  gathered  by  an 
adaptive  optics  system  similar  in  concept  to  the  AMOS  system.  The  actual  per¬ 
formance  and  design  of  the  adaptive  optics  system  is  not  treated  here.  Advanced 
applications  of  Artificial  Intelligence,  Pattern  Recognition.  Neural  Networks  and 
other  advanced  computer  science  techniques  are  not  treated  here.  Due  to  the  se¬ 
curity  classification  of  the  bulk  of  AMOS  imagery,  experimentation  was  limited  to 
unclassified,  simulated  imagery  (see  Chapter  3).  Subjective  evaluation  of  imagery 
by  experienced  Photo-Interpreters  is  beyond  the  scope  of  this  thesis. 

1.5  Summary  of  Key  Results 

This  research  supports  the  hypothesis  that  automatic  objective  image  quality 
measurement  is  possible  for  digital  images  of  space  objects.  The  two  recommended 
image  quality  metrics  (IQMs)  presented  here  are 

_  P^pptr 

IQM^Z  T,  (1-5) 

0  P~Plowr 

and 

IQM  =  var{f,^i{x,y)]  (1.6) 
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where  F{p,  9)  is  the  discrete  Fourier  Treinsform  of  an  image,  piower  and  pepper  are  the 
practical  band  limits  of  the  human  visual  system,  fsoi>e:{x,  y)  is  a  Sobel  edge  enhanced 
image  and  vor[-  •  ■]  is  sample  variance.  These  algorithms  were  implemented  in  the 
IDL  software  package. 

The  metrics  were  tested  and  validated  against  a  database  of  raw  images  gen¬ 
erated  by  a  software  simulation  of  an  atmospheric  imaging  adaptive  optics  system. 
Since  the  simulator  allows  precise  control  (and  knowledge)  of  seeing  conditions  and 
target  brightness,  which  directly  impact  image  quality,  changes  in  image  quality 
metric  values  could  be  directly  and  objectively  compared  to  changes  in  these  simu¬ 
lation  parameters.  As  simulated  brightness  or  seeing  conditions  were  quantitatively 
improved,  a  successful  IQM  should  give  a  quantitatively  higher  image  quality  rating. 

Also,  recall  that  the  modified  inverse  filtering  operation  discussed  above  re¬ 
quires  the  user  to  select  the  optimal  noise  suppression  filter  width.  The  image  qual¬ 
ity  metrics  were  tested  against  a  database  of  images  reconstructed  with  a  variety 
of  noise  suppression  filter  widths  to  give  some  indication  of  the  feasibility  of  using 
these  metrics  for  totally  un-supervised  image  reconstruction. 

In  all  test  cases  the  metrics  performed  consistently,  detecting  absolute  changes 
in  image  quciiity  simulation  parameters  and  noise  filter  widths.  An  operational  eval¬ 
uation  of  metric  performance  was  not  performed  in  this  research.  Such  evaluation 
would  necessarily  include  comparing  photo-interpreter  assessment  to  automated  im¬ 
age  quality  metric  rating,  on  actual  imagery  instead  of  simulated  imagery. 

1.6  Document  Organization 

Chapter  2  will  discuss  the  current  knowledge  concerning  un-supervised  objec¬ 
tive  image  quality  assessment  as  found  in  the  image  processing  and  optics  literature, 
and  present  the  candidate  Image  Quality  Metrics  that  were  successfully  tested. 

Chapter  3  will  discuss  how  the  Image  Quiility  Metrics  were  tested  on  “raw” 
image  ensembles  using  simulation  software. 
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Chapter  4  will  discuss  the  results  of  the  testing,  A  feasibility  argument  is  also 
presented  regarding  the  use  of  the  Image  Quality  Metrics  to  automatically  select  the 
optimum  width  of  the  triangular  reconstruction  noise  filer,  mentioned  previously  in 
this  chapter. 

Chapter  5  will  discuss  conclusions  and  any  further  research  that  may  be  indi¬ 
cated. 
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II.  Image  Quality  Metrics 


Since  the  goeil  of  this  research  is  measuring  relative  image  quality,  a  reasonable 
first  step  would  be  to  obtain  two  different  quality  images  of  the  Scime  object  and 
compare  and  contrast  various  quantities  associated  with  the  images.  If  there  are  any 
differences  found,  the  next  step  would  be  to  determine  if  these  differences  appear 
consistently  for  all  images  of  different  quality,  and  then  determine  of  this  effect  is 
consistent  for  a  variety  of  different  targets.  Choosing  quantities  to  be  tested  is  the 
subject  of  this  chapter,  and  the  testing  of  those  quantities  is  covered  in  Chapters  3 
and  4. 

In  the  image  processing  literature,  images  are  generally  analyzed  in  the  spatial 
domain  (i.e.  analysis  of  characteristics  and  distribution  of  th^  actual  pixels)  or  in 
some  transform  domain,  where  the  image  is  represented  as  the  coefficients  of  some 
linear  combination  of  orthogonal  basis  functions  (or  basis  images)  (6j.  Section  1 
discusses  image  quality  measurement  with  the  most  commonly  used  of  these  trans¬ 
forms,  the  Fourier  Transform,  mentioned  in  Chapter  1.  Section  2  discusses  Image 
Qucility  Metrics  derived  from  spatial  domain  quantities  of  the  image. 

2.1  The  Fourier  Transform  and  Image  Quality 

In  Chapter  1  the  Linear  Systeras/Fourier  Optics  model  of  image  propagation 
waf  discussed.  It  was  mentioned  in  passing  that  the  u  and  v  frequency  variables 
are  .spatial  frequency  variables.  This  implies  that  the  higher  the  spatial  frequency, 
the  faster  things  are  changing  in  the  image.  This  alone  might  intuitively  lead  one 
to  think  that  a  “good”  (i.e.  highly  resolved,  good  detail)  image  could  have  a  larger 
amount  of  spectral  content  in  the  higher  spatial  frequencies  than  a  “bad"  image, 
since  “detail”  implies  that  gray  levels  are  changing  rapidly  across  the  image  as  the 
structure  of  the  target  is  revealed  (3). 
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Also,  spectral  content  in  the  very  low  frequencies  would  generally  imply  simple, 
large-area  changes  in  background  level  (3).  Therefore,  the  very  low  spatial  frequencies 
might  not  have  a  serious  impact  on  the  amount  of  target  detail  that  an  image  reveals. 

The  working  hypothesis  in  this  section  is  that  “good'’  image  spectra  differ  in 
some  quantifiable  and  consistent  manner  from  “bad”  image  spectra.  The  intuitive 
arguments  presented  above  lead  to  the  specific  hypothesis  that 

{high  frequency  spectral  content)  {''good!'  image).  (2.1) 

If  this  hypothesis  can  be  supported,  then  an  Image  Quality  Metric  (IQM)  such  as: 

^2^  rPupper 

/  /  \F{p,e)\dpd9  (2.2) 

‘'0  Plover 

is  implied  {F  is  the  image  Fourier  Tiansform  discussed  in  Chapter  1,  where  u  and 
V  Cartesian  frequency  coordinates  have  been  changed  to  p  and  6  polar  frequency 
coordinates,  p  =  or  the  Euclidean  distance  from  zero  frequency).  There  is 

support  in  the  literature  for  such  a  hypothesis,  as  discussed  next. 

2.1.1  Human  Visual  System  Transfer  Function.  The  idea  that  the  human 
visual  system  (HVS)  is  more  receptive  to  some  spatial  frequencies  and  less  receptive 
to  others,  as  implied  above,  is  widely  ciccepted  (6|  and  can  be  readily  translated  into 
a  linear  systems  vocabulary.  SpecificaJly,  the  underlying  hypothesis  is  that  the  HVS 
can  be  modelled  as  a  spatial  frequency  transfer  function  [10]  [11]  [12]  [19]. 

Under  certain  simplifying  assumptions^,  a  relatively  simple  first  order  approx¬ 
imation  of  the  Human  Visual  System  as  a  magnitude  transfer  function  (MTF)  has 
been  developed.  The  relevant  research  involved  experimentation  with  human  ob¬ 
servers  who  evaluated  images  with  periodic  patterns  of  different  fundamental  fre- 

'  Assumptions  such  as;  the  gray  level  differences  are  limited  to  the  approximately  linear  portion 
of  HVS  dynamic  range;  the  observer  can  look  at  the  image  for  as  long  as  is  preferred;  the  observer 
is  at  the  optimal  distance  from  the  image  representation. 
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quencies.  By  noting  the  observers’  sensitivity  across  a  variety  of  spatial  frequencies, 
an  approximation  to  the  frequency  response  of  the  HVS  could  be  estimated  (11). 

The  researchers  found  that  the  HVS  is  insensitive  to  all  but  gross  changes  at 
very  low  spatial  frequencies.  Also,  like  all  optical  systems,  the  HVS  is  band-limited, 
and  insensitive  to  higher  frequency  structuie  in  an  image.  (Note  that  there  would 
also  be  bandwidth  constraints  imposed  by  the  optics  and  sampling  of  the  imaging 
system.)  Taken  together,  the  conclusion  is  that  the  HVS  transfer  function  can  be 
approximated  Jis  a  lotationally  symmetric  bandpass  system,  with  one  possible  cross 
section  shown  in  Figure  2.1.  Analytical  fits  to  this  curve  such  as: 

HVS  Tronsfer  Function 


\H(p)\  =  (A  +  Bp)e<^'  (2.3) 

are  often  used,  where  A,B,and  C  are  determined  empirically  for  each  application, 
and  depend  on  image  display,  observer  distance  from  display,  image  sampling  and 
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array  size  [11],  This  idea  that  the  KVS  can  be  modelled  as  a  bandpeiss  filter  will 
feature  prominently  in  the  effort  to  measure  image  quality,  as  discussed  next. 

2.1.2  Spectral  Image  Quality  Metric.  How  can  this  model  be  applied  to 
image  quality  measurement?  Observe  the  two  images  of  a  Russian  Okean  satellite 
shown  in  Figure  2.2.  These  images  were  obtained  from  an  adaptive  optics  simulation 


Figure  2.2  Okean  Satellite,  r,,  =5  (left)  and  15  (right)  cm  respectively 


described  in  Chapter  3.  The  imaging  conditions  are  alike  in  every  respect  except 
seeing  conditions:  r<,  =  5  cm  on  the  left  and  15  cm  on  the  right.  It  is  clear  that  the 
second  image  reveals  more  detail  about  the  target. 

How  do  the  spectra  of  the  two  images  compare?  First  of  all,  to  make  a  mean¬ 
ingful  comparison,  the  energy  of  the  two  images  must  be  normalized: 


fnormi^y  V} 


y) 

Si  Sy  forigi^i  P) 


(2.4) 


where  J„,g{x,y)  is  the  original  image.  Also,  since  depicting  the  comparison  of  2 
two-dimensionad  surface  plots  is  difficult,  it  would  be  convenient  (at  least  for  the 
purpose  of  presenting  the  concepts)  to  raidially  average  the  modulus  the  two  spectra: 


S' radavgip)  ~ 


y:9\s{p,9)\ 


number  of  pixels  at  distance  p 


(2.5) 


where  \F{p,6)\  is  the  modulus  of  the  discrete  two-dimensional  Fourier  Transform  of 
fnoTm{x,y)  (again  in  polar  coordinates  with  p  -  y/u^  +  v^,  the  Euclidean  distance 
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in  pixels  -  rounded  to  the  nearest  integer  -  from  the  zero  frequency  origin).  Now 
Fradavg[p)  is  a  rctil  onc-dimensional  array  which  can  be  plotted  as  a  curve  instead  of 
a  surface. 

The  radially-averaged  spectra  of  the  two  example  images  are  shown  in  Figure 
2.3.  Note  that  the  curve  corresponding  to  the  good  image  spectrum  is  greater  than 
the  curve  corresponding  to  the  bad  image  spectrum  for  a  specific  midrange  band  of 
frequencies.  Chapter  4  will  show  that  this  effect  occurs  consistently  for  a  wide  variety 
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Figure  2.3  Spectral  Comparison  of  Different  Quality  Images 

of  targets  and  image  degradation  effects  tested  in  this  research,  a  vitally  important 
result.  The  same  result  is  documented  in  (lOj.  This  result  is  consistent  with  our 
model  of  the  HVS  as  a  bandpass  system,  since; 
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•  the  AMOS  mission  involves  the  high  resolution  imaging  of  space  objects  for 
consumption  by  human  analysts; 

•  the  Fried  Parameter  r^,  is  a  direct  indication  of  the  queility  (resolution)  of  an 
AMOS  image  for  this  application. 

Therefore  a  larger-r^,  image  should  have  higher  frequency  content  in  the  HVS  band- 
pciss  frequencies. 

Since  the  the  area  (or  volume  in  the  non-radially  averaged  spectrum)  under  the 
curve  is  different  for  images  of  different  quality  (within  a  certain  frequency  range), 
the  following  candidate  spectral  Image  Quality  Metric  (IQM)  is  suggested: 

Puppet 

!QM  =  ‘£  •£  |f(^,«)|  (2.6) 

0  P — Ptower 

where  the  piower  and  Pup^er  are  determined  empirically.  The  selection  of  these  limits 
will  be  discussed  in  Chapter  4.  Note  that  this  formulation  is  in  effect  integrating 
(summing)  the  magnitude  spectrum  after  filtering  by  a  rectangular  cross-section 
bandpass  filter.  This  rectangular  bandpass  filter  could  be  regarded  as  a  simpler 
approximation  of  the  HVS  bandpass  shown  in  Figure  2.1. 

A  similar  quantity  to  the  above  IQM  is  suggested  in  [12].  The  major  difference 
there  is  that  instead  of  using  the  rectangular  filter  approximation  as  above,  the  image 
is  filtered  with  an  analytical  curve-fit  of  Figure  2.1  as  well  aa  a  filter  to  dead  with 
oblique-angle  effects  in  aerial  remote  sensing  (which  doesn’t  apply  to  this  thesis). 

The  Intertw:tive  Data  Language  (IDL)  software  package  implements  the  so- 
called  “Feist"  Fourier  Transform  (FFT)  adgorithms,  facilitating  easy  implementation 
of  this  metric.  This  metric  was  tested  (see  Chapter  3)  and  its  performance  is  dis¬ 
cussed  in  Chapter  4. 
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2.2  Histograms  and  Image  Quality 

The  histogram  of  an  image  is  a  common  spatial  domain  image  processing  and 
anaJysis  tool.  The  histogram  of  an  image  is  simply  a  tally  of  the  number  of  pic¬ 
ture  elements  (pixels)  that  have  a  certain  gray  level,  plotted  against  gray  level: 
histogram  =  n(fc)  where  n{k)  is  the  number  of  pixels  with  gray  level  =  k.  The  his¬ 
togram  bin  sizes  are  generally  the  same  as  the  gray-level  quantization  of  the  imaging 
system,  making  the  histogram  a  discrete  array  of  integers.  Normalizing  the  curve 
n{k)  to  unit  area  yields  a  first  order  estimate  of  the  probability  density  function 
(PDF)  of  the  image:  p{k)  =  ^  {(oi  N  X  N  array)  (6]  [4). 

The  working  hypothesis  here  is  that  images  of  different  quality  have  consistent, 
quantifiable  differences  in  their  histograms. 

2.2.1  Histogram-Based  Image  Quality  Metrics.  Consider  again  the  two 
satellite  images  in  Figure  2.2.  After  normalizing  the  energy  of  the  first  image  to  be 
the  same  as  the  second  image^,  or 


*  » 

(2.7) 

^  Z)  fiix,  y) 

X  V 

(2.8) 

.normV**'*  if/  ^ 

(2.9) 

(where  /i.2(i,  y)  are  the  respective  images  in  Figure  2.2)  the  histograms  of  both  of 
these  images  are  calculated  (Figures  2.4. 

A  distinct  difference  in  the  shape  of  the  histogram  curves  is  evident.  Observe, 
for  example,  that  the  good  image  histogram  has  entries  at  the  higher  gray  levels, 
while  the  bad  image  histogram  does  not.  This  indicates  that,  even  though  both 

^This  allows  energy  normalization,  while  keeping  convenient  integer  histogram  bin  sizes,  as 
opposed  to  normalizing  to  unit  energy,  where  all  gray  levels  and  bin  sizes  would  be  fractional. 
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Histogrom 


images  have  the  same  totai  energy,  the  better  image  has  some  brighter  pixels.  Also 
note  that  the  better  image  has  more  zero  pixels.  This  is  because  the  bad  image, 
which  is  of  poorer  resolution,  has  its  image  energy  “smeared  out”  across  the  image, 
yielding  non-zero  (gray)  pixels  in  the  worse  image  that  were  (zero)  black  in  the  better 
image. 


2. 2-1.1  Histogram  Features  and  Image  Quality.  The  above  obser¬ 
vations  are  all  somewhat  subjective.  The  goal  is  to  somehow  extrcict  parameters 
from  these  histograms  that  captures  these  observations  in  a  way  that  correlates  with 
image  quality.  As  a  starting  point,  note  the  following  common  parameters,  called 
histogram  features,  which  are  often  used  to  characterize  image  histograms  [6]: 


HISTOGRAM  PARAMETERS: 

Moments 

”1.  = 

Absolute  Moments 

m.  =  ZlZo  l^rp(*) 

Central  Moments 

P.  =  LkZo  - ’^iYpW 

Absolute  Central  Moments 

A.  =  Ekio  ii'  -  ^irp(^) 

Entropy 

«  =  -n:o-p(‘)iogjpW 

where  gmax  is  the  maximum  gray  level  in  the  histogram.  These  quantities  are  not 
unique  to  image  processing,  but  are  also  used  in  general  statistics  applications.  As 
such,  some  of  these  parameters  are  better  known  by  other  names: 


first  absolute  centrsd  moment,  p,i 

dispersion 

second  central  moment,  H2 

variance 

second  moment,  m? 

mean  square  value 

third  central  moment,  ^3 

skewness 

fourth  central  moment,  ^4 

kurtosis 

Note  that  some  of  these  features  can  be  measured  without  explicitly  determining  the 
histogram: 

"1.  =  E  E y)  (210) 

*  V 
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*  y 

(2.11) 

(2.12) 

X  y 


Consider  again  the  two  example  histogram  curves.  Obviously,  the  shapes  of 
the  curves  cire  significantly  different.  It  is  plausible  that  the  moments  of  these  curves 
can  provide  a  quantitative  indication  of  the  aspects  of  histogram  shape  that  may 
be  relevant  to  image  quality.  Variance,  for  example,  as  mentioned  in  [6]  can  give 
an  indication  of  “activity” ,  in  the  gray  level  amplitudes  .  This  is  the  same  intuitive 
argument  mentioned  in  the  previous  section  of  this  chapter  as  a  motivation  to  inves¬ 
tigate  spatial  frequency  quality  metrics.  The  working  hypothesis  here  is  that  some 
or  all  of  these  histogram  features  correlate  with  AMOS  image  quality. 

2.2.2  Histogram  Features  as  Structure  Factors.  Linfoot  [8]  presents  similar 
intuitive  arguments  when  he  redefines  some  of  the  above  histogram  features  as  follows 
for  an  energy  normalized  image: 


second  moment, 

structural  density  factor 

variance,  H2 

normalized  mean  structural  density 

dispersion,  /ti 

Jx.y  y)dxdy 

such  that  c{x,y)  =  "structure  function"  or  "contrast  function" 

-  f{x,y)  -m 

He  relates  these  factors  to  the  “peakiness”  or  vairiability  of  image  amplitudes,  and 
the  amount  of  structure  or  “fine  object  detail”  an  image  contains.  Thus,  support  for 
the  hypothesis  exists  in  optics  literature. 


2.2.3  Histogram  Features  as  Sharpness  Functions.  Buffington  and  Muller  [2] 
also  present  some  of  the  histogram  features  given  above,  but  from  an  entirely  dif¬ 
ferent  context,  using  an  idea  they  call  image  sharpness.  The  authors’  goal  was 
to  derive  algorithms  (called  sharpness  functions)  that  produce  a  maximum  output 
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when  phase  distortion  at  the  telescope  image  plane  is  at  a  minimum.  Their  eventual 
goal  is  to  use  image  “sharpness”  measurements  to  drive  adaptive  optics  hardware 
in  a  feedba.ck  loop,  providing  real-time  maximization  of  the  sharpness  function,  and 
therefore,  real-time  correction  of  image  phase  distortion. 

Consider  one  shcirpness  function  given  in  [2]; 

$3  =  f  f  M{x,y)f{x,y)dxdy  (2.13) 

J  Jimagt 

where  f{x,y)  is  the  received  image  and  M{x,y)  is  the  target  image,  known  a  priori. 
Now,  since  it  assumes  prior  knowledge  of  the  target,  it  cannot  be  used  operationally 
at  AMOS,  but  it  is  instructive  to  consider  it  for  illustration  purposes. 

The  sharpness  function  above  is  similar  to  the  correlator /matched- filter  concept 
in  digital  communication  theory.  Consider  a  digitaJ  communications  receiver  which 
receives  a  waveform  that  has  been  corrupted  by  noise  or  other  propagation  effects. 
The  receiver’s  purpose  is  to  find  out  which  symbol  was  sent.  One  common  approach 
is  to  cross-correlate  the  received  waveform  with  a  collection  of  stored  symbols,  and 
select  the  symbol  that  gave  the  highest  cross-correlation  value^.  Buffington  <ind 
Muller  are  using  nearly  the  same  concept  in  their  scheme  of  image  correction. 

Now,  if  the  target  is  not  known  a  priori  (e.g.  AMOS  operations),  then  the 
target  function  M{x,y)  in  S3  can  be  replaced  with  f{x,y),  the  received  image,  as  in 
the  BufRngton-Muller  sharpness  function  given  by 


or 


•^1  =  /  /  fix,y)f{x,y)dxdy 
J  Jtmage 


Si=  f  f  f^{x,y)dxdy 

J  J  image 


(2,14) 


(2.15) 


^For  two  real  one-dimensional  functions  f{x)  and  M(x),  cross-correlation  is  defined  asr/M(x)  = 
fa  +  a)da  and  the  autocorrelation  of  /(z)  is  defined  as  r//(z)  =  /(o)/(z  -f  rt)<ia. 
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which  is  the  second  moment,  and  aJso  the  autocorrelation  peak  0),  of  the 

image.  Buffington  and  Muller  prove  that  this  function  is  also  maximized  when 
telescope  phase  aberration  is  minimized.  Similar  conclusions  are  reached  for  the 
third  and  fourth  moment  functions. 

Three  different  arguments  have  now  been  cited  as  motivation  for  investigating 
various  statistical  moments  of  an  image  with  respect  to  image  quality.  Of  all  the 
moments  mentioned  above,  only  the  second  centred  moment  will  be  given  further 
consideration  here.  This  is  due  to  its  intuitive  connection  to  the  idea  of  variance, 
where  variance  of  image  amplitudes  generally  imparts  image  information.  Prelimi¬ 
nary  testing  early  in  this  research  effort  concluded  that  the  first,  third  and  fourth 
moment  performed  nearly  the  same  as  variance  regarding  the  relative  measurement 
of  image  quality. 

Also  note  that  there  is  no  real  basis,  other  than  convenience,  for  choosing 
between  the  second  moment  (sharpness)  and  the  second  central  moment  (variance) 
as  a  measure  of  relative  image  quality  when  the  images  have  been  energy-normalized. 
This  is  because  var{/)  s=  E\f^]  -  (£'(/|)^  (where  var[-  •  •]  is  the  variance  operation 
and  •  •)  is  the  expectation  operation)  and  if  the  image  f  is  always  normalized 
to  the  same  value,  E\f]  and  therefore  (£’(/))^  will  always  be  the  same  value,  so 
the  variance  and  sharpness  will  always  differ  by  the  same  constant.  Image  sample 
variance  is  therefore  arbitrarily  chosen  here  for  further  investigation  as  an  image 
quality  metric. 

2.2.4  Histogram  Features,  Information  Theory,  and  Entropy.  Entropy  is 
an  important  concept  in  many  areas  of  digital  signal  processing,  and  since  it  is  often 
intuitively  related  to  the  information  content  of  a  signal,  it  is  a  natural  choice  for 
a  candidate  image  quality  metric.  As  will  be  seen  in  Chapter  4  however,  it  did  not 
prove  to  be  a  consistent  indicator  of  image  quality.  As  a  rule,  unsuccessful  image 
quality  metrics  are  not  discussed  in  this  thesis,  but  it  was  felt  that  entropy  is  an 


important  enough  quantity  that  it’s  testing  and  failure  in  this  research  effort  should 
be  explicitly  noted. 

Entropy,  as  originally  introduced  in  the  field  of  statistical  mechanics,  is  a  mea¬ 
sure  of  the  number  of  microscopic  states  by  which  a  given  macroscopic  state  can 
be  realized.  If  there  are  N  possible,  equally  likely  microscopic  states,  the  entropy  is 
given  by 

H  =  lnN  =  -  In^  =  -  Inp  (2.16) 

v/here  p  would  be  the  probability  of  each  state.  If  the  states  are  not  equally  likely, 
but  each  have  probability  p{k),  then  the  entropy  is  an  average  [9]; 

(-lnp(A:))  = -52p(*)lnp(^)-  (2.17) 

k 

Shannon  provided  the  mathematical  basis  for  modern  communications  theory 
when  he  applied  the  entropy  concept  to  messages  and  information,  replacing  thermo¬ 
dynamic  micro-states  with  message  symbols  (20].  In  this  formulation,  information 
imparted  by  a  symbol  in  a  message  depends  on  the  number  and  probability  of  each 
particular  symbol.  For  image  processing  applications,  the  gray  level  of  a  pixel  plays 
the  role  of  a  symbol.  Typically,  in  information  theory  applications  to  digital  data, 
the  above  natural  logarithm  is  usually  replaced  by  log2  which  is  eiccomplisbed  by 
simply  dividing  by  In  2.(20] 

Consider  a  256  X  256  pixel  eight  bit  image,  where  there  are  65536  elements 
which  can  each  take  on  256  values.  If  the  pixels  were  all  the  same  vcilue,  the  image 
would  convey  little  information,  since,  practically  speaking,  there  would  be  no  image. 
Only  one  bit  of  information  per  symbol,  versus  8  (2®  =  256  gray  levels),  would  be 
needed  to  couvey  the  information  in  the  image.  An  image  with  8  gray  levels  could 
reveal  quite  a  bit  more  detml  about  different  brightnesses  at  different  parts  of  the 
target.  This  image  might  require  at  least  three  bits  per  symbol  (2®  =  8  gray  levels) 
to  convey  the  information  in  the  image.  The  trend  that  these  examples  are  meant  to 
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show  here  is  that  an  image  with  more  “information  content’’,  or  entropy,  has  a  wider 
and  flatter  histogram.  In  fcict,  entropy  is  maximized  for  a  “flat"  or  uniform  PDF, 
or  histogram.  More  generally,  entropy  provides  some  measure  of  histogram  shape. 
Also,  since  entropy  gives  a  quantitative  indication  of  the  “uncertainty”  of  a  message 
there  an  intuitive  argument  for  investigating  entropy  as  an  Image  Quality  Metric. 


2.S.5  Histogram- Based  Image  Quality  Metrics  with  Spatial  Filtering.  Con¬ 
sider  the  second  moment  image  quality  metric  mentioned  previously: 


Si  =  [  f  f{x,y)dxdy  =  rff{0,0) 

J  Jtmage 


(2.18) 


By  Parseval’s  Theorem  this  quantity  is  equal  to 


JJjFM? 


dudv 


(2.19) 


or  in  discrete  form: 

(2-20) 

U  V 

This  expression  is  similar  in  concept  to  the  spectral  image  quality  metric  from  the 
first  section  of  this  chapter: 

Puppt* 

IQM  =  Y^  £  \F{p,0)\.  (2.21) 

In  both  cases  the  two  preceding  equations  are  adding  the  modulus  (or  modulus 
squared)  of  Fourier  coefficients  of  an  image. 

Notice  however,  that  the  second  moment  is  the  sum  of  all  image  energy  regard¬ 
less  of  spatial  frequency.  This  ignores  the  idea  presented  previously  that  energy  in 
very  low  spaticU  frequencies  is  relatively  unimportant  to  image  quality.  This  leads  to 
the  following  question:  given  that  the  human  visual  system  (HVS)  eicts  as  a  band¬ 
pass  filter,  would  it  be  desirable  (and  possible)  to  also  “filter”  the  images  in  the 
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spatial  domain  before  applying  the  histogram/image  moment-based  IQMs,  as  was 
done  with  the  spectral  IQM? 

Part  of  the  answer  to  this  question  lies  in  the  fact  that  histograms  (and  image 
moments  derived  from  them)  contain  no  pixel  position  information.  One  could  take 
a  nearly  perfect  image  of  the  highest  qucJity  and  randomly  rearrange  the  pixels  into 
a  meaningless  jumble  and  the  histogram  and  the  IQMs  based  on  it  would  remain 
unchanged.  Another  aspect  to  consider  is  that  noise  manifests  itself  as  random 
variations  in  image  pixel  values.  The  worst-case  scenario  is  that  one  of  the  candidate 
spatial  IQMs  could  possibly  indicate  that  an  image  with  high  noise  was  of  higher 
quality  than  an  image  with  low  noise,  just  the  opposite  of  what  is  desired. 

It  is  possible  to  approximate  the  effect  of  a  frequency-domain  filter  in  the 
spatial  domain.  This  is  known  as  spatial  filtering.  The  basic  approach  is  to  replace 
a  pixel  value  with  some  linear  combination  of  the  value  of  itself  and  it’s  neighbors. 
This  can  be  visualized  as  sliding  a  mask:  (Figure  2.5)  across  the  image  and  replacing 


W1 

W2 

W3 

W4 

W5 

W6 

W7 

W8 

W9 

Figure  2.5  Spatial  Filter  General  Mask 

the  pixel  at  the  center  of  the  mask  with,  for  example,  the  sum  of  the  products  of 
the  mask  values  and  the  pixel  values  under  the  mask.  In  Figure  2.5,  for  example,  if 
the  values  of  the  pixels  under  the  mask  are  Pi,P2,  -  ■  • 9,  then  the  pixel  p^  becomes 

Pb.new  =  w’lPi  +  ^2P2  H - +  ^9P9-  Note  that  pixel  position  information  now  plays  a 

factor  in  the  new,  spatirilly-filtered  image.  Note  also  that  the  Fourier  Transform  of 
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the  image  is  not  needed  to  perform  the  filtering,  implying  a  significant  computational 
savings  in  most  cases  [4]. 


One  of  the  most  important  categories  of  spatial  filters  are  known  as  edge  de¬ 
tection  opemtors  [4].  The  mask  of  the  Sobel  edge  detection  operator  is  shown  in 
Figure  2.6.  These  two  masks  are  both  applied  to  the  image,  one  after  the  other. 


Figure  2.6  Sobel  Operator  Masks 


There  are  two  masks  because  of  the  nature  of  the  gradient  operation  upon  which  the 
Sobel  Operator  is  based; 

The  Sobel  Operator  is  known  as  agrcidient  filter  because  it  replaces  the  originail  image 
with  an  estimate  of  the  gradient  of  the  image.  See  Figure  2.7  for  two  examples  of 
Sobel  edge-enhanced  images. 

The  gradient  provides  an  indication  of  the  location  and  intensity  or  sharpness 
of  the  edges,  or  discontinuities,  in  an  image,  which  usually  impart  the  only  useful 
information  in  an  image  to  the  observer.  The  Sobel  Operator  produces  low  output  for 
uniform  image  regions  (i.e.  low  spatial  frequencies),  and  is  also  somewhat  insensitive 
to  noise  effects  (i.e.  random  high  frequency  content)  [4]  .  Also,  the  HVS  itself 
provides  a  form  of  edge  detection  [6].  This  implies  that  the  Sobel  Operator  acts  afi  a 
type  of  spatial  HVS  filter.  This  research  will  investigate  whether  or  not  the  variance 
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SOBEL  OPERATOR 


Figure  2.7  Sobel  Operator  Examples  -  Left:  Original  Images  ~  Right:  Sobel  Filtered 
Images 

image  quality  metric  presented  above  will  perform  better  if  the  image  is  first  Sobel 
filtered. 

Note  that  this  idea  hcis  some  support  in  the  literature.  One  of  the  proposed 
sharpness  functions  proposed  in  [2]  is  the  second  moment  of  a  partial  derivative  of 
the  image,  or: 

(2-23) 

where  f{x,y)  is  the  image.  This  formulation  is  another  approach  to  finding  the 
variance  of  the  edge  information  of  an  image.  Another  source  (7)  suggests  analyzing 
the  histogram  shape  of  a  Sobel-filtcred  image  as  a  method  of  automatically  detecting 
flaws  in  abrasive  diamond  film  as  it  is  being  fabricated. 

2.3  Summary 

The  following  IQMs  are  to  be  tested,  with  results  presented  in  Chapter  4; 

1.  Spectral  Content  (Band  Limited)  of  Image 
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2.  Variance  of  Originad  Image 

3.  Variance  of  Sobel  Filtered  Image 
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III.  Research  Approach 


This  chapter  will  discuss  the  methods  used  to  test  the  various  in  ige  quality 
metric  candidates. 

3.1  Adaptive  Optics  Simulation  Software 

Michciel  Roggemann  of  the  Air  Force  Institute  of  Technology  has  developed 
FORTRAN  software  which  simulates  a  general  adaptive  optics  imaging  system  (16). 
The  user  provides  a  “true”  image  as  input  to  the  system,  say  a  CAD  drawing  of  a 
satellite,  such  as  the  Russian  Okean  satellite  model  shown  in  Figure  3.2,  and  defines 
the  simulation  imaging  parameters.  The  parameters,  and  some  typical  values  used 
in  this  research  are  shown  below: 


SIMULATION  PARAMETERS: 

number  of  frames 

100 

1 

density  of  WFS  detectors 

10  across  diameter 

telescope  mirror  diameter 

1.0  meters 

wavelength  of  light 

500  nanometers 

target  distance 

500  kilometers 

average  number  of  photons  per  image 

varied,  4th  to  8th  magnitude 

average  number  of  photons  per  WFS  subaperture 

varied,  4th  to  8th  magnitude 

actuator  spacing 

11  centimeters 

atmosphere  coherence  diameter,  r<, 

1 

varied,  7  to  13  cm 

RMS  detector  reewd  noise 

15  photons  per  pixel 

The  astronomical  magnitude  scale  mentioned  above  is  defined  such  that  the  if  the 
total  radiation  flux  between  two  stars  is  given  by  the  ratio  si/sj,  their  magnitudet 
differ  by  mi  -  =  -2.512  logio(si/«2).  a  previously  agreed  upon  flux  level  for 

m  =■  0.  For  example,  the  star  Vega  has  an  apparent  magnitude  of  0.14.  If  two  objects 
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differ  in  absolute  magnitude  by  1,  the  number  of  photons  leaving  the  objects  differ 
by  a  factor  of  2.512.  If  they  differ  by  5  magnitudes,  the  number  of  photons  differs  by 
a  factor  of  100.  Roggemann  [17]  has  numerically  ccdculated  the  number  of  photons 
reaching  an  earth-bound  adaptive  optics  telescope  (through  a  typiccd  atmosphere): 


MAGNITUDE 

No.  Photons  at  Wavefront  Sensor 

No.  Photons  at  Image  Plane 

4 

207 

10654 

5 

83 

4262 

6 

33 

1705 

7 

13 

682 

8 

5 

273 

A  block  diagram  of  the  major  software  simulator  functions  is  shown  in  Figure  3.1. 


Figure  3.1  Block  Diagram  Description  of  aptive  Optics  Simulation  Software 

3-2 


Two  set?  of  output  ensemble  averages  are  shown  below,  first  with  magnitude 
held  constant  and  tq  varying,  then  with  tq  held  constant  and  magnitude  varying 
(Figures  3.3  and  3.4),  and  sil  other  simulation  parameters  held  constant  throughout 
both  cases.  Note  the  different  effects  of  the  atmosphere  and  light  level  on  resolution 
and  quality. 


Figure  3.2  Okean  Satellite  CAD  Drawing 


Figure  3.3  Okean  Satellite,  4th  magnitude,  =  13,  10,  and  7  cm  respectively 


Figure  3.4  Okean  Satellite,  =  13cm,  4th,  5th,  and  6th  magnitudes  respectively 

Besides  the  Okean  satellite  image,  two  other  test  objects  were  used  in  the  sim¬ 
ulation:  (Figure  3.5)  This  variety  of  shapes,  structures,  and  periodicities  is  assumed 
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Figure  3.5  Eorsat  and  Rorsat  Imaging  Target  CAD  Drawings 

to  provide  a  reasonable  test  of  image  quality  measures  as  applied  to  space  object 
imagery. 

3.2  Testing  Parameters 

The  optical  propagation  simulator  gives  the  user  direct  control  over  three  very 
important  factors  which  have  a  direct  bearing  on  AMOS  image  quality,  but  which 
cannot  be  known  in  advance  in  real-world  operations: 

1.  Target  Magnitude  (light  level); 

2.  Seeing  Conditions  (Fried  Parameter);  and 

3.  The  original  target  irradiance  function. 

Since  the  goal  is  to  detect  differences  in  image  quality,  the  actual  values  of  the  other 
parameters  are  somewhat  unimportant,  provided  they  are  held  constant  while  light 
level  and  seeing  conditions  are  varied.  For  testing  of  raw  imagerj’,  the  most  straight¬ 
forward  procedure  is  to  generate  a  large  number  of  images  for  ail  three  targets, 
varying  magnitude  and  rp,  calculate  the  various  IQMs  for  these  images,  and  plot  the 
IQM  values  against  both  magnitude  and  tq. 

As  noted  above,  tq  is  a  direct  indication  of  image  resolution.  Therefore,  a 
valid  IQM  should  increase  monotonically  as  a  function  of  tq.  Also,  object  magnitude 
impacts  image  quality  through  its  effect  on  the  performance  of  the  adaptive  optics 


system  (WFS  measurement  errors),  and  through  its  effect  the  image  signal-to-noise 
ratio.  A  valid  IQM  should  decrease  monoionically  as  a  function  of  magnitude. 

One  other  test  parameter  will  be  considered  in  this  research.  When  deal¬ 
ing  with  image  coding,  compression  and  transmission,  fidelity  parameters  are  com¬ 
monly  used,  which  provide  a  measure  of  how  closely  some  “original”  image,  /orty(^.  y) 
matches  a  “final”  image  /(x,y)  after  undergoing  some  process.  One  common  fidelity 
measure  is  root-mean-square  error. 

Crm,  =  -4  U,g{x,  y)  -  f{x,  y)  gray  levels.  (3.1) 

U  X  y 

It  is  well  known  that  such  fidelity  measures  are  not  good  indicators  of  image  qucdity 
as  evaluated  by  the  HVS.  In  other  words,  the  HVS  is  known  to  be  more  toler¬ 
ant  of  certain  types  of  errors,  and  more  sensitive  to  others  [4].  For  instance,  in 
Figures  3.3  and  3.4  two  different  types  of  image  corruption  effects  are  noticeable, 
depending  on  the  parameter  being  varied,  yet  Cmt  would  provide  no  information  on 
which  type  was  more  or  less  objectionable  to  a  human  observer.  However  in  both 
cases,  Crmi  would  still  increase  with  decreasing  image  quality.  Therefore  it  may  be 
instructive  to  parameterize  IQMs  against  e^m**  <-0  provide  a  coarse  validity  check  of 
the  results.  It  would  be  expected  that  valid  IQMs  should  be  a  generally  decreasing 
function  of  e^mj- 

IQM  performance  on  reconstructed  (inverse  filtered)  imagery  can  «dso  be  pa¬ 
rameterized  against  the  triangular  noise  filter  width.  However,  this  filter  width  does 
not  have  a  simple,  easily  quantifiable  effect  on  image  quality.  If  the  filter  is  too 
narrow,  the  reconstructed  image  will  be  a  low  quality,  low  resolution  “blob”.  If  the 
noise  filter  is  too  wide,  noise  effects  will  lower  the  image  quality.  One  possible  test 
approach  would  be  to  reconstruct  a  single  ensemble  with  a  variety  of  filter  widths, 
calculate  IQMs  for  these  various  output  images,  and  plot  IQM  versus  noise  filter 
width.  One  would  expect  a  convex  curve  with  maximum  corresponding  to  the  opti- 
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mum  filter  width.  It  would  fall  to  the  observer  to  determine  subjectively  if  this  filter 
width  is  indeed  optimum. 

The  results  of  this  testing  <ue  discussed  in  Chapter  4. 
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IV.  Results 


4-1  Introduction 

This  chapter  investigates  the  hypothesis  that  the  Image  Quality  Metrics  (IQMs) 
presented  in  Chapter  2  correlate  ^dth  atmospheric  seeing  conditions  (rj,),  target  mag¬ 
nitude,  and  image  fidelity  (root-mean-square  error),  as  controlled  by  the  adaptive 
optics  simulator  discussed  in  Chapter  3.  A  key  assumption  is  that  these  parame¬ 
ters  are  indeed  indicative  of  Image  Quality  as  experienced  by  AMOS  operators  and 
customers  using  real-world  data.  Key  illustrative  results  are  shown  here,  with  the  re¬ 
mainder  in  the  appendix.  This  chapter  also  examines  the  idea  of  using  the  IQMs  for 
reconstructed  image  selection,  specifically,  selection  of  a  reconstruction  noise  filter. 

In  all  cases  a  successful  IQM  will  consistently  and  monotonically  change  with  a 
quantifiable  change  in  or  target  magnitude,  as  controlled  by  the  image  simulator. 
The  two  dimensional  surface  plots  shown  in  the  following  two  sections  of  this  chapter 
will  plot  To  on  one  axis,  target  magnitude  on  the  second  axis,  and  the  IQMs  eis  a 
function  of  the  two.  The  IQMs  should  give  the  high-resolution,  high  brightness 
images  represented  by  one  corner  of  the  plot  domain  the  highest  quality  rating  and 
the  low- resolution,  low  brightness  images  represented  by  the  opposite  corner  of  the 
plot  domain  the  lowest  quality  rating.  Similarly,  any  cross  section  taken  parallel  to 
the  To  or  magnitude  axes  should  give  a  monotonically  increasing  quality  rating  curve 
as  either  r<,  goes  up  or  magnitude  goes  down. 

Metrics  were  also  parameterized  against  RMS  error.  Recall  from  Chapter  3 
that  RMS  error  only  correlates  approximately  with  image  quality  as  perceived  by 
the  human  visual  system.  However,  observing  the  relation  between  IQMs  and  RMS 
error  should  prove  to  be  an  instructive,  qualitative,  overall  check  on  the  validity  of 
the  other  results. 
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4-2  Spectral  Image  Quality  Metrics 

Recall  from  Chapter  2  that  the  band  limits  for  the  spectral  metric  are  best 
determined  empirically.  This  is  because  the  display,  image  frame  size,  and  the  spatial 
bandwidth  of  the  particular  telescope  system  in  use  are  all  system  dependent  and 
all  effect  the  image  spectra  as  sampled  by  the  FFT  adgorithm.  Figure  4.1  shows 
an  example  of  how  this  was  accomplished  for  the  simulated  imagery  used  in  this 
research.  A  simple,  practical  approach  for  band  limit  selection  is  adopted  here: 
since  the  goal  is  discrimination  between  images  based  on  quality,  observe  where 
the  radially  averaged  spectra  of  different  quaUty  images  intersect.  As  mentioned  in 
Chapter  2,  the  area  under  the  cur/e  segments  between  these  two  intersection  points 
is  sensitive  to  image  quality,  so  these  two  intersection  points  are  a  practical  indication 
of  the  HVS  passband.  This  passband  can  be  determined  by  inspection. 

The  radially  averaged  magnitude  spectra  (normalized  to  unit  area)  are  shown 
for  5  images  of  the  Okean  satellite  (Figure  4.1).  In  this  case,  the  Fried  P<u-am- 
eter  was  held  constant,  and  the  target  magnitude  was  varied.  Note  that  for  this 
image  format,  for  radial  frequency  distances  of  about  1  to  15  samples  of  normal¬ 
ized  frequency,  the  curves  corresponding  to  any  particular  image  are  greater  than 
the  curves  corresponding  to  a  lower  quality  image.  A  similar  effect  is  found  for 
the  second  plot,  where  the  magnitude  was  held  constant  and  the  Pried  Pajameter 
was  varied  (Figure  4.2).  Determining  the  frequencies  at  which  the  radially  averaged 
magnitude  spectra  of  different  quality  images  overlap  seems  to  be  a  reasonable  func¬ 
tional  method  of  determining  the  HVS  passband.  For  this  particular  image  format 
and  (simulated)  telescope  system,  the  band  limit  was  therefore  chosen  to  go  from  3 
to  11  pixels  rcidial  frequency  distance.  This  type  of  empirical  determination  would 
have  to  be  accomplished  by  inspection  for  any  different  image  format  or  telescope 
system.  The  IQM  algorithm  merely  adds  up  the  magnitude  spectrum  pixels  falling 
within  this  distance  range  from  the  zero  frequency  origin. 
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Figure  4.1  Radially  Averaged  Magnitude  Spectra,  Vg  =  10  cm,  5  Different  Magni¬ 
tudes,  Okean  Satellite  Image 
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A  representative  surface  plot  from  the  Okean  t<uget  spectral  IQM  experiment 
is  shown  in  Figure  4.3.  Cross-sections  of  this  surface  plot  are  shown  in  Figures 


Figure  4,3  Spectral  IQM  Parameterization,  Okean  Satellite  Image 


4.4  and  4.5. 

This  IQM  gives  similar  results  for  the  other  two  test  objects,  as  shown  in 
the  appendix.  The  metric  is  monotonicajly  increasing  with  and  with  brightness 
(decreasing  with  magnitude).  This  appears  to  be  a  successful  Image  Quality  Metric. 

As  a  validity  check,  the  Quality  metric  is  plotted  vs.  RMS  error.  Note  the 
approximate  trend  of  decreasing  quality  rating  with  increasing  RMS  error.  This  leads 
weight  to  the  hypothesis  that  this  is  a  valid  quality  metric.  Recall,  as  mentioned 
previously,  that  fidelity  metrics  like  RMS  error  correlate  only  approximately  with 
HVS-perceived  image  quality. 
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Spectrol  lOM  ^  Spectral  IQM 


30  40  50  60 

Image  RMS  Error 

Figure  4.6  Spectral  IQM  vs  RMS  Error,  Okean  Satellite  Image 


J^.8  Histogram- Based  Image  Quality  Metrics 

A  representative  surface  plot  from  the  Okean  target  Variance  IQM  experiment 
is  shown  in  Figure  4.7.  Cross-sections  of  this  surface  plot  are  shown  in  Figures 
4.4  and  4.5. 

The  metric  performs  reasonably  well  for  brighter  objects,  but  as  the  objects 
get  dimmer  that  6th  magnitude,  the  performance  curve  slope  changes  sign.  Observe 
Figure  4.10,  which  shows  the  Okean  satellite  with  =  7cm  and  magnitudes  of  6,7, 
and  8  respectively.  Judging  from  the  curve  in  Figures  4.7  ,  4.8  and  4.9,  the  two 
lower  quality  images  have  “spoofed”  the  variance  metric  into  categorizing  them  as 
better  than  the  higher  quality  6th  magnitude  image  (Figure  4  7).  As  discussed  in 
Chapter  2,  pixel  position  information  is  not  captured  by  the  histogram  moments. 
The  7th  and  8th  magnitude  images  shown  do  indeed  have  more  “variance”  than 
the  6th  magnitude  image,  but  this  increased  variance  is  not  due  any  higher  detail 
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Voriance  IQM 


60 


2  4  6  8  10  12  14 

Fried  Porometer 

Figure  4.8  Variance  IQM  vs  Fried  Parameter,  Okean  Satellite  Image 
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Variance  IQM 


Magnitude 

Figure  4.9  Variance  IQM  vs.  Target  Magnitude,  Okean  .satellite  Image 


Figure  4.10  Okean  Satellite,  =  7cm  and  magnitudes  of  6,7,  and  8  Respectively 
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or  meaningful  structural  content  of  these  images.  It  is  instead  due  to  the  lower 
eigneil-to-noise  ratio  of  the  dimmer  images. 

Observe  how  the  two  lower  cuality  images  in  question  have  little  in  the  way  of 
edges  or  discontinuities.  It  seems  re?.8onable  that  if  a  Sobel  operator  were  applied  to 
these  images  and  then  the  variance  were  calculated,  relative  variance-based  quality 
rating  of  these  images  would  be  reduced,  since  there  is  less  energy  content  in  these 
images  that  would  survive  the  Sobel  operation.  A  parameter  plot  of  the  Sobel- 
Variance  IQM  is  shown  in  Figures  4.11,  4.12  and  4.13,  verifying  this  hypothesis. 


AMOS  users  could  determine  whether  the  Sobel  extension  of  the  variance  met¬ 
ric  is  necessary.  If  the  signal-to-noise  ratio  rarely  drops  low  enough  to  spoof  the 
variance  IQM,  the  Sobel  operation  may  be  unnecessary.  However,  it  costs  little  in 
the  way  of  computation  time,  and  the  algorithm  exists  in  “canned"  form  in  the  IDL 
software  package. 
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Sobel-Varionce  lOM  g  Sobel-Vorionce  IQM 


2  4  6  8  10  12  14 

Fried  Parometer 


e  4.12  Sobel- Variance  IQM  vs  Fried  Parameter,  Okean  Satellite  Image 


Mognitude 


Figure  4.13  Sobel- Variance  IQM  vs  Target  Magnitude,  Okean  Satellite  Image 
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Although  there  are  now  indications  that  variance  <done  is  not  a  consistent 
indicator  of  image  quality,  it  is  instructive  to  include  it  in  discussions  which  follow, 
for  comparison  with  the  Spectral  and  Sobel-Variance  IQMs. 

The  RMS  parameter  plots  for  the  regular  and  Sobel  variance  IQMs  are  shown 
in  Figures  4.14  and  4.15  An  interesting  feature  of  these  plots  is  the  two  “arms”  that 


split  off  from  the  curve  at  high  RMS  error  levels.  The  upper  arm  corresponds  to  the 
low  signal-to-noise  ratio  as  the  images  get  dimmer  and  dimmer.  The  lower  arm  of  the 
plot  corresponds  to  the  image  blurring  as  r,  gets  smaller  and  smaller.  Notice  that  if 
the  degradation  were  only  related  to  seeing  conditions,  the  Sobel  extension  to  this 
metric  would  not  be  necessary.  The  implication  is  that  degraded  seeing  conditions 
do  not  increase  variance. 
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RMS  Imoge  Error 

RMS  error,  Okean  Satellite  Image 
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The  final  histogram  based  metric  discussed  was  entropy.  This  metric  would  be 
expected  to  give  a  lower  number  as  image  quality  increased  [2]  (1).  This  turned  out 
to  be  the  case  when  only  seeing  condition  were  varied  (Figure  4.16). 


Figure  4.16  Entropy  vs.  r„  (cm),  Okean  Satellite  Image 


However  the  entropy  metric  responded  in  the  opposite  manner  as  the  magni¬ 
tude  of  the  image  increased  (became  dimmer).  Figure  4.17  shows  this  for  To  =  10 
and  13  cm. 

The  conclusion  is  that  entropy  is  not  a  consistent  metric,  since  it  gives  higher 
quality  ratings  for  lower  SNR  images.  Since  image  degradations  caused  by  low 
magnitude  (low  SNR)  do  not  occur  separately  from  degradations  caused  by  seeing 
conditions,  this  metric  is  operationally  unacceptable.  Although  entropy  is  function¬ 
ally  related  to  histogram  shape,  this  relationship  is  apparently  not  reliable  in  terms 
of  prissing  image  queility  information.  Preliminary  investigations  into  measuring  the 
entropy  of  edge-enhanced  images  also  also  showed  the  same  inconsistency  as  above, 
showing  higher  quality  for  lower  SNR. 
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Figure  4.17  Entropy  vs.  Magnitude,  Okean  Satellite  Image 

Although  all  unsuccessful  metrics  are  not  presented  in  this  thesis,  entropy  is  an 
important  theoretical  concept  throughout  a  wide  variety  of  engineering  disciplines. 
It  is  also  a  logical  candidate  to  consider  for  quality  measurement,  since  it  intuitively 
relates  to  “information  content”  of  signals,  which  is  itself  one  possible  definition  of 
“quality".  For  this  reason,  it  merits  special  mention. 

4-4  Reconstructed  Image  Selection 

From  an  observing  session  of  a  low  earth  orbit  space  object,  AMOS  customers 
usually  require  two  images  as  the  object  climbs  from  ascending  horizon  to  zenith, 
one  image  captured  as  the  object  is  approximately  near  zenith,  and  two  more  as 
the  object  ascends  to  the  other  horizon.  This  means  dozens  of  50  to  100  exposure 
ensembles  (each  of  which  is  reconstruct 'd  to  form  a  single  output  image)  to  choose 
from. 
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Furthermore,  each  image  reconstruction  via  inverse  filtering  is  reconstructed 
with  a  number  of  triangular  noise  filter  widths.  As  noted  in  Chapter  1,  since  the 
OTF  of  the  telescope  system  is  a  random  variable  through  time,  the  optimum  width 
of  this  filter  is  best  determined  empirically.  This  means  reconstructing  the  image 
with  a  number  of  filter  widths  and  subjectively  selecting  the  best  product.  This  last 
aspect  of  image  product  selection  is  investigated  here.  Any  conclusions  drawn  could 
also  be  applied  to  the  selection  problem  mentioned  in  the  previous  paragraph. 

It  is  expected  that  very  narrow  filter  widths  would  result  in  output  images 
that  were  low-resolution,  low-quality  “blobs”,  since  the  high  frequency  detail  would 
be  filtered  away.  Very  wide  filter  widths,  on  the  other  hand,  could  be  expected  to 
allow  in  high  frequency  noise  content,  also  lowering  image  quality.  Therefore,  some 
intermediate  filter  width  would  be  desired.  The  hypothesis  to  be  examined  here  is 
that  plotting  an  IQM  as  a  function  of  noise  filter  width  would  produce  a  convex 
curve,  with  the  maximum  of  the  curve  corresponding  to  the  optimimum  filter  width. 

First  consider  the  reconstructed  satellite  images  shown  in  Figure  4.18.  The 
source  ensemble  is  100  exposures  of  the  Okean  target,  with  an  of  5  centimeters 
and  practically  infinite  (one  million  photons  per  frame)  brightness.  This  means  high 
signal-to-noise-ratio.  The  IQM  values  in  this  experiment  for  the  Spectral,  Variance, 
and  Sobel  Variance  IQMs  are  shown  in  Figure  4.19.  In  this  experiment,  the  Spectral 
IQM  rates  the  third  image,  with  filter  radius  12,  as  the  best.  The  Sobel-Variance 
IQM  rates  the  fourth  image,  with  filter  radius  16  as  the  best.  The  Variance  IQM 
rates  the  6th  image,  with  filter  rcidius  20,  as  the  best. 

The  next  experiment  is  with  the  same  target,  but  with  =  10  centimeters, 
and  a  target  magnitude  of  6  (Figure  4.20).  The  IQM  values  in  this  experiment  for 
the  Spectral,  Variance,  and  Sobel  Variance  IQMs  are  shown  in  Figure  4.21.  Again, 
the  peaks  of  each  of  the  three  curves  can  be  picked  off  the  graph.  The  Spectral 
IQM  picked  the  third  image,  with  uoise  filter  radius  of  12,  as  the  best  image.  The 
Sobel-Variance  Metric  picked  the  second  image,  with  noise  filter  radius  of  8,  as  the 
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P'gure  4.18  Okean  Target  Reconstructed  Images,  High  SNR,  =  5cm,  Noise  Filter 
Base  Radii  =  4,8,12,16,20,24,28,  and  32  Pixels  Respectively,  From  Left 
to  Right,  Top  to  Bottom 
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IQM  Response  os  o  Function  of  Reconstruction  Noise  Filter  Width 


Bose  Rodius  of  Triongulor  Noise  Filler 


Figure  4.19  IQM  Selection  of  Noise  Filter  Radius,  Okean  Target  Reconstructed 
Images,  High  SNR,  To  =  5cm 
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Figure  4.20  Okean  Target  Reconstructed  Images,  6th  magnitude,  =•  10cm,  Noise 
Filter  Base  Radii  =  4,8,12,16,20,21,28,  and  32  Pixels  Respectively, 
From  Left  to  Right,  Top  to  Bottom 
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IQM  Response  as  a  Function  of  Reconstruction  Noise  Filter  Width 


8ose  Rodius  of  Trionguior  Noise  Filter 


Figure  4.21  IQM  Selection  of  Noise  Filter  Radi,  Okean  Target  Reconstructed  Im 
ages,  6th  magnitude.  To  =  lOcra 


best  image.  Notice  this  time  that  the  Variance  IQM  es  a  result  significantly 
diflferent  from  the  other  two  IQMs,  selecting  the  image  with  noise  filter  radius  of 
28.  This  is  because  noise  effects  h.  ve  been  introduced  into  the  experiment,  due  to 
the  low  target  magnitude.  As  mentioned  earlier  in  this  chapter,  the  Variance  IQM 
is  “spoofed”  by  high  hovels  of  noise  in  the  image.  This  may  be  an  explanation  of 
the  result  shown  in  Figure  4.21,  where  the  Variance  IQM  is  selecting  reconstructed 
images  with  significantly  higher  noise  levels  than  the  other  two  IQMs.  Notice  also, 
that  even  though  has  increased  from  its  value  in  the  first  experiment,  which 
would  effectively  increase  the  bandwidth  of  the  image,  the  optimal  filter  radius  for 
the  Spectral  and  Sobel- Variance  metrics  decreased,  because  of  the  need  to  keep  out 
the  noise  that  was  not  present  in  the  first  experiment. 

The  third  and  final  experiment  presented  in  this  section  is  a  5th  magnitude 
Eorsat  target,  with  =  7  centimeters.  The  reconstructed  images  are  shown  in 
Figure  4.22,  and  the  Ii,^M  results  are  shown  in  Figure  4.23.. 

Notice  in  Figures  4.20  and  4.21,  for  example,  when  significant  noise  is  present, 
that  the  variance  metric  is  again  fooled  by  noise.  In  all  the  cases  shown  above. 
Spectral  and  Sobel  Variance  IQMs  react  quite  similarly,  and  appear  to  not  be  spoofed 
by  noise  artifacts.  Surface  plots  parameterizing  these  two  IQMs  (Spectral  and  Sobel- 
Variance)  against  To  and  noise  filter  width  are  shown  below.  Notice  that  these  plots 
show  that  when  magnitude  (and  signal-to-noise-ratio)  is  changed,  optimum  filter 
width  is  changed.  Optimal  filter  width  also  changes  when  (and  the  effective 
telescope  bandwidth)  is  changed. 

Notice  that  the  performance  of  the  metrics  compared  to  tq  and  magnitude  on 
reconstmefed  images  is  similar  to  the  performance  on  “raw”  ensembles,  so  these  met¬ 
rics  are  equally  valid  for  comparing  either  raw  or  reconstructed  imagery,  assuming 
in  reconstructed  image  case  that  comparisons  are  only  between  images  that  used 
triangular  noise  filters  o  he  same  width.  This  conclusion  can  be  drawn  immedi- 
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Figure  4.22 
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Eorsat  Target  Reconstructed  Images,  5th  magnitude,  r„  =  7cm,  Noise 
Filter  Base  Radu  =  4,8,12,16,20,24,28,  and  32  Pixels  Respectively, 
From  Left  to  Right,  Top  to  Bottom 
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IQM  Response  as  a  Function  of  Reconstruction  Noise  Filter  Width 


Base  Rodius  of  Triongulof  Noise  Filter 


Figure  4.23  IQM  Selection  of  Noise  Filler  Radius,  Eorsat  Target  Reconstructed 
Images,  5th  magnitude,  To  =  7cm 


Figure  4.24  Spectral  IQM  vs.  r^,  and  Noise  Filter  Radius,  Okean  Satellite  Image, 
Magnitude  =  4 


Figure  4.25  Spectral  IQM  vs.  Target  Magnitude  and  Noise  Filter  Radius,  Okean 
Satellite  Image,  =  10  cm 
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Figure  4.26  Sobel- Variance  IQM  vs.  and  Noise  Filter  Radius,  Okean  Satellite 
Image,  Magnitude  =  4 


Figure  4.27  Sobel- Variance  IQM  vs.  Target  Magnitude  and  Noise  Filter  Radius, 
Okean  Satellite  Image,  =  10  cm 


ately  since  it  is  agreed  that  and  magnitude  are  themselves  immediate,  monotonic 
indicators  of  image  quality. 

However  when  using  the  IQMs  for  reconstructed  imagery  with  different  noise 
filter  widths,  such  as  in  the  plots  above,  notice  that  different  metrics  select  different 
filter  widths  as  “best”.  The  user  of  the  IQM  will  have  to  decide  which,  if  any  of 
these  metrics  are  to  be  used  when  comparing  imagery  of  different  noise  filter  widths. 

1^.5  Summary  of  Results 

On  all  images  tested,  the  Spectral  and  Sobel- Variance  Image  Quality  Met¬ 
rics  give  indications  of  being  successful  Image  Quality  Metrics  for  both  raw  and 
reconstructed  images.  The  Spectral  IQM  cannot  be  implemented  in  a  particular 
system  until  the  appropriate  band  limits  are  empirically  determined,  as  discussed 
above.  The  Sobel* Variance  IQM  requires  no  such  analysis,  and  provides  practically 
the  same  capability  for  un-supervised  discrimination  of  images  based  on  quality. 

The  Spectral  and  Sobel- Variance  IQMs  have  been  conceptually  validated  on  a 
database  of  raw  satellite  imagery.  A  feasibility  argument  hM  also  been  presented  for 
using  these  two  IQMs  to  select  optimum  reconstruction  noise  filter  bandwidth.  The 
two  IQMs  can  give  slightly  different  results,  in  which  case  a  subjective  assessment  is 
required  to  decide  which  IQM  to  use  for  this  purpose. 


V.  Conclusions  and  Recommendations 

5.1  Thesis  Summary 

This  thesis  project  has  dealt  with  the  problem  of  “measuring”  the  quality  of  a 
digital  image,  specifically  a  digital  image  of  an  orbiting  satellite,  as  observed  from  a 
ground-based,  adaptive-optics  telescope,  through  the  Earth’s  atmosphere. 

The  challenge  of  this  problem  lies  in  the  well-established  fact  that  “image 
quality”  is  very  much  in  the  eye  of  the  beholder.  It  is  often  taken  for  granted  that 
the  best  assessment  of  image  quality  necessarily  includes  a  subjective  evaluation  by 
the  person  who  will  be  using  the  imagery.  This  is  a  very  serious  restriction  if  a 
quality  assessment  is  required  in  an  image  processing  application. 

Consider,  for  example,  the  thesis  sponsor,  the  Air  Force  Maui  Optical  Station, 
AMOS,  which  captures  high  resolution  imagery  of  orbiting  space  objects.  During 
a  particular  observing  session,  hundreds  of  raw  image  frames  may  be  gathered  for 
post  processing.  Some  of  these  images  are  of  lower  quality  than  others,  and  it  is 
assumed  desirable  to  exclude  them  from  post  processing.  In  order  to  do  this,  the  hu¬ 
man  operator  must  manually  observe  each  of  the  raw  image  frames  and  subjectively 
decide  which  images  to  exclude.  Similarly,  when  using  post  processing  to  digitally 
reconstruct  the  imagery,  there  are  a  variety  of  post-processing  techniques  available, 
and  the  techniques  ail  include  user  defined  variables  to  be  adjusted  based  on  trial- 
and-error  euid  subjective  judgement.  The  user  must  subjectively  select  the  optimum 
reconstruction  technique  and  parameters. 

If  there  were  a  way  for  the  computer  to  assign  a  “quality  number”  to  the 
images,  the  above  processes  could  conceivably  be  fully  automated.  The  selections 
of  best  and  worst  images  could  be  reduced  to  a  simple  thresholding  the  the  quality 
number. 
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This  thesis  investigated  a  variety  of  quantities  which  a  computer  could  au¬ 
tomatically  extract  from  a  digital  image,  testing  whether  they  correlate  with  image 
quality,  as  defined  for  AMOS  imagery.  The  two  most  successful  image  quality  metrics 
(IQMs)  were 

1.  A  spectra/ IQM  which  involved  measuring  the  spectral  (Fourier  doraaiia)  content 
of  the  image  within  the  passband  of  the  human  visual  system;  and 

2.  A  Sobel  Variance  IQM  which  involved  finding  the  sample  variance  of  a  Sobel- 
enhanced  image. 

It  was  shown  that  through  Parseval’s  theorem  these  two  metrics  are  conceptually 
related. 

It  is  important  to  recall  that  metrics  can  only  be  meaningfully  compared  if 

•  all  images  are  of  the  same  object 

•  that  object  is  at  relatively  the  same  orientation  (i.e.  from  the  same  short 
time-span  during  an  AMOS  pass 

•  all  images  are  normeuized  to  the  same  total  energy. 

Also,  the  human  visual  system  passband  is  most  easily  and  practically  determined 
by  inspection  as  discussed  in  Chapters  2  and  4. 

These  metrics  were  tested  against  a  database  of  satellite  images  which  were 
created  by  an  adaptive  optics  software  simulation.  This  computer  simulation  allowed 
the  user  to  quantitatively  specify  the  two  parameters  which  physically  determme 
AMOS  image  quality:  target  brightness  and  atmospheric  seeing  conditions.  A  library 
of  images  for  three  typical  space  objects  was  created,  while  systematically  varying 
these  two  simulation  quality  parameters.  Any  prospective  IQM  could  then  be  tested 
against  this  image  library.  A  successful  IQM  would  be  expected  to  give  consistent, 
quantifiable  changes  in  image  quality  ratings  as  these  simulation  parameters  changed 
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value.  The  Spectral  and  Sobel- Variance  IQMs  gave  consistent  and  reasonable  results 
for  this  test  database  of  images. 

After  the  validity  of  the  IQMs  was  tested  on  raw  imagery,  the  problem  of 
reconstructed  image  selection  was  examined.  One  of  the  most  common  image  recon¬ 
struction  techniques  used  at  AMOS  is  a  modified  inverse  filtering  approach,  requiring 
a  user-specified  noise  attenuation  filter.  Some  of  the  above  library  images  were  re¬ 
constructed  with  a  variety  oi  noise  filter  widths  and  then  evaluated  with  the  Spectral 
and  Sobel- Variance  IQMs.  The  results  show  that  these  IQMs  might  reasonably  be 
used  to  automatically  select  the  optimum  reconstruction  parameters.  Since  the  two 
metrics  sometimes  gave  different  results,  the  user  might  best  determine  which  of  the 
two  acquires  the  best  track  record  on  actual,  operational  imagery,  in  the  course  of 
adopting  these  concepts  into  a  fully  automated  image  reconstruction  system. 

5. 2  Further  Research 

The  inverse  filtering  reconstr  ct-on  process  can  often  cause  noise  “artifacts"  to 
be  introduced  into  the  final  image.  '  .•'me  sort  of  texture-based  segmentation  of  these 
noisy  reconstructed  images,  to  remove  these  artifacts  might  increase  the  validity  of 
the  IQMs  in  reconstructed  image  selection.  Using  the  IQMs  themselves  on  sraeill 
subregions  of  the  image  might  even  prove  to  be  a  good  method  of  segmentation.  Of 
course,  with  segmentation  there  is  always  the  risk  of  accidentally  eliminating  valid 
image  information. 

It  has  also  been  suggested  that  instead  of  approximating  the  human  visual  sys¬ 
tem  MTF  as  a  .simple  rectangular  bandpass,  the  more  exau:t  exponential  formulation 
shown  in  Chapter  2  might  be  adopted,  to  give  more  satisfaictoiy  results  in  recon¬ 
struction  parameter  selection.  The  task  of  determining  the  human  visual  system 
MTF  for  a  particular  imaging  and  display  system  would  necessarily  become  more 
problematic  in  this  case. 
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Another  source,  [10],  has  also  suggested  calculating  the  geometric  moments  the 
band-limited  power  spectrum  of  cin  as  an  image  quality  measure: 

iQM  =  r  r*""  p"F(p,  e)dp  (5.1) 

Je=o  Jp=Q 

or  in  discrete  form 

Pmpptr 

(5.2) 

«  P^-O 

where  n  is  2,3,  or  4.  This  could  also  prove  to  be  a  valid  metric,  judging  from 
Figure  2.3  in  Chapter  2,  since  the  “mass  distributions”  of  a  good  image  spectrum 
and  a  bad  image  spectrum  are  distinctly  different.  Note  that  if  this  turned  out  to  be 
a  valid  IQM,  it  would  not  have  the  complication  of  requiring  the  user  to  determine 
the  actual  humaii  visual  system  MTF.  This  IQM  formulation  was  not  tested  in  this 
thesis. 

Another  possible  avenues  of  research  could  involve  modifying  the  Sobel-Variance 
IQM  by  using  other  common  edge  enhancement  filters  such  as  the  Roberts  or  Prewitt 
operators. 

5.3  Ccnclusion 

Ou  all  images  tested,  the  Spectral  and  Sobel-Variance  Image  Quality  Metrics 
give  indications  of  being  successful  Image  Quality  Metrics  for  both  raw  and  recon¬ 
structed  images.  Of  course,  the  hypothesis  that  the  Spectral  and  Sobel-Variance 
IQMs  are  valid  measures  of  image  quality  can  never  be  absolutely  proven  per  se, 
but  this  thesis  has  lent  weight  to  the  hypothesis  by  showing  that  it  has  not  been 
disproven  for  raw  and  reconstructed  imagery  for  an  interesting  variety  of  satellite 
shapes.  Therefore,  the  objectives  of  this  research  have  been  satisfied. 
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Spectrol  (QM.  RORSAT  Image 


Figure  A. 3  Spectral  IQM  vs  Fried  PMameter,  EORSAT  Satellite  Image 
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Sobel-Vorionce  lOM,  RORSiAT  Image  Spectra!  lOM,  EORSAT  Image, 


Magnitude 


Figure  A. 4  Spectral  IQM  vs  Target  Magnitude,  EORSAT  Image 
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Figure  A. 6  Sobel- Variance  IQM  vs  Target  Magnitude,  RORSAT  Image 


Figure  A. 7  Sobel- Variance  IQM  vs  Fried  Parameter,  EORSAT  Image 
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Sobel -Vorionce  lOM,  EORSAl 


Figure  A  8  Sobe!-Variarice  IQM  vs  Target  Magnitude,  EORSAT  Image 
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